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Abstract. A random walk on a directed graph gives a Markov chain on the 
vertices of the graph. An important question that arises often in the context of 
Markov chain is whether the uniform distribution on the vertices of the graph is 
a stationary distribution of the Markov chain. Stationary distribution of a Markov 
chain is a global property of the graph. In this paper, we prove that for a regular 
directed graph whether the uniform distribution on the vertices of the graph is a 
stationary distribution, depends on a local property of the graph, namely if (u,v) 
is an directed edge then outdegree(u) is equal to indegree(v). 

This result also has an application to the problem of testing whether a given 
distribution is uniform or "far" from being uniform. This is a well studied prob- 
lem in property testing and statistics. If the distribution is the stationary distri- 
bution of the lazy random walk on a directed graph and the graph is given as 
an input, then how many bits of the input graph do one need to query in order 
to decide whether the distribution is uniform or "far" from it? This is a problem 
of graph property testing and we consider this problem in the orientation model 
(introduced by Halevy et al). We reduce this problem to test (in the orientation 
model) whether a directed graph is Eulerian. And using result of Fischer et al. on 
query complexity of testing (in the orientation model) whether a graph is Eule- 
rian, we obtain bounds on the query complexity for testing whether the stationary 
distribution is uniform. 



1 Introduction 

Markov chains are one of the most important and most studied structures in Theoretical 
Computer Science. The use of Markov chains in other fields like Physics, Biology, Eco- 
nomics, Statistics etc. cannot be over emphasised. The most important characteristics 
of a Markov chain are its stationary distribution and its mixing time. It is very important 
to understand the structure of stationary distribution of a Markov chain. To be precise, 
one often wants to know if a particular distribution is a stationary distribution of a given 
Markov chain. Usually a stationary distribution of a Markov chain is a global property 
of the graph, hence whether a particular distribution is a stationary distribution of a 
Markov chain depends on the global structure of that graph. 

One of the most commonly used Markov chains in Theoretical Computer Science 
is the Markov chain obtained by a random walk on a directed graph. In this paper, we 
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focus on this kind of Markov chain. We prove that contrary to normal perception, for 
this kind of Markov chain, if the graph is regular then whether the uniform distribution 
is a stationary distribution depends on a local property of the graph. The following 
theorem, which is the main result of this paper (see section |3), is a statement about that 
local property. 

Theorem 1. If~& = (V, EJ) is a digraph such that the total degree ( that is Indegree(v) + 
Outdegree(v)) for every vertex v G V is the same, then the uniform distribution on the 
vertices ofG is a stationary distribution (for the Markov chain generated by a random 
walk on ~u) if and only if the graph has the following properties: 

1. For all v £ V, Indegreejv) and Outdegree(v) ^ 0, 



2. For every edge (u,v) G E, Outdegree(u) = Indegree(v). 

As an application of this result, we want to test whether the uniform distribution 
over the vertices is the stationary distribution for the Markov chain generated by a lazy 
random walk-on G . 

1.1 Application to Property Testing of Distributions 

In property testing, the goal is to look at a very small fraction of the input and distin- 
guish whether the input has a certain property or it is "far" from satisfying the prop- 
erty. If ~& = (v, E) is a digraph such that the total degree (that is Indegree(v) + 
Outdegree(v)) for every vertex v G V is the same, then we consider the Markov chain 
obtained by the lazy random walk on vertices of G . Thus, the distribution can be given 
implicitly by the directed graph. 

Suppose that the underlying undirected graph is known beforehand and for every 
edge there is an unit cost to know its orientation. Now, we want to distinguish between 
the following two cases: 

- whether the uniform distribution on the vertices is the stationary distribution of the 
Markov chain. 

- whether more than e fraction of edges has to be reoriented such that the result- 
ing stationary distribution of the Markov chain is the uniform distribution on the 



The goal is to distinguish between the above cases while incurring the minimum cost, 
that is, by knowing the orientation of minimum number of edges. This model for testing 
graph properties is called the orientation model [4|. This model has been well studied 
in the field of property testing. We show that the above problem can be reduced to an 
earlier studied problem of testing (in orientation model) whether a directed graph is 
Eulerian. We use the results of [3 ] to obtain algorithm that incurs sub-linear cost for the 
above problem. We present this part of our result in section 

1 A lazy random walk (starting from a particular vertex) on a directed graph is a random walk 
in which at each time step, the walk stays where it is with probability \ or moves according to 
the usual random walk. Moreover, it converges to a unique stationary distribution. 




vertices. 
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2 Preliminaries 

2.1 Graph Notations 

Throughout this paper, we will be dealing with directed graphs (possibly with multiple 
edges between any two vertices) in which each edge is directed only in one direction. 
To avoid confusion, we will call them oriented graphs because each edge is oriented 
and is not bidirectional. We will denote the oriented graph by = (V, E ) and the 
underlying undirected graph (that is when the direction on the edges are removed) by 
G = (V, E). For a vertex v G V, the in-degree and the out-degree of v in are denoted 
by d~(v) and d + (v) respectively. An oriented graph = (V, ~E*) is called a degree-Z\ 
oriented graph if for all v € V, d~ (v) + d + (v) = A. In this paper, we will be focusing 
on degree-Z\ oriented graphs. 

2.2 Markov Chains 

A Markov chain is a stochastic process on a set of states given by a transition matrix. 
Let S be the set of states with l^l = n. Then, the transition matrix T is a n x n matrix 
with entries from positive real; the rows and columns are indexed by the states; the u, v- 
th entry T u<v of the matrix denotes the probability of transition from state u to state v. 
Since T is stochastic, J2 V must be 1. A distribution fj, : S — >• M + on the vertices is 
said to be stationary if for all vertices v, 



If G is an oriented graph then a random walk on G defines a Markov chain, where, 
the states are the vertices of the graph; the probability to traverse an edge is given by 
the quantity p U)V = d ^ u ^ ; and hence, the transition probability T Ujt) from vertex u to 
vertex v is p u>v times the number of edges between u and v. 

In this paper, we will only consider the Markov chains that arise from random walks 
on oriented graphs. We will call them random-walk-Markov-chain on G, where ~G* 
is an oriented graph. The uniform distribution on the vertices of ~cH is a stationary dis- 
tribution for this Markov chain if and only if for all v G V, 



3 Structure of Graphs with Uniform Stationary Distribution 
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The following Theorem is a rephrasing of Theorem [T] 
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Theorem 2. Let = (V, E) be a degree- A oriented graph, then the uniform distri- 
bution is a stationary distribution if and only if for all v G V, both d" (v), d + (y) ^ 
and for all (it, v) G ~E*, 

d + {u) = d~(v) 

Proof: First of all, note that the uniform distribution is a stationary distribution for , 
iff for all v G V 

^ ^ Pu.v 1 ^ ^ Pv.w-j 

u:(u,v)£E* w:(v.w)£E* 

where p u v is the transition probability from vertex u to vertex v. Since we are doing a 
random walk on an unweighted oriented graph, for all (u, v) G i^, p u . v = d +^ u ^ ■ 

Thus, if the graph has the property that for all (u, v) G d + (u) = d~ (v), then 
note that 



d+{u) ^ d~(v) ' 

u:(u,v)GE u:(u,v)£E u:(u,v)£E 

the last equality holds because the summation is over all the edges entering v (which is 
non-empty) and thus have d~ (v) number of items in the summation. Similarly, 

w:(v,w)G E w:(v,w)£ E 

Thus, if the graph has the property that for all (it, v) G i?, d + (u) = d~ (v), then the 
uniform distribution is a stationary distribution for the Markov chain. 

Now let us prove the other direction, that is, let us assume that the uniform dis- 
tribution is a stationary distribution for the Markov chain. Note that, if the uniform 
distribution is a stationary distribution then there is a path from u to v if and only if u 
and v are in the same strongly connected component of C^. This is because the uniform 
distribution is a stationary distribution if and only if for every cut C = Vi U V2 where 
V-2 = (F\Vi),we have 

' Pu.v = ^ Pu,v 

(u,v)<ee*, and ueVi,v£V 2 (u.v)e~^, and u<ev 2 ,v£Vi 

In other words, if a stationary distribution is uniform then it implies that every connected 
component in the undirected graph is strongly connected in the directed graph. 

Let vq, v\, V2, ■ ■ ■ , Vt be a sequence of vertices such that the following conditions 
are satisfied: 



- For all i > 0, (v l+ \,v l ) G E 

- For all z > 0, d + (v 2 i+i) = min jd+(u>) : (w, i>2i) G is* j 



Testing Uniformity of Stationary Distribution 5 



- For all i > 0, d + (v2i ) = max 




We call such a sequence as "degree-alternating" sequence of vertices. 

Claim 3 Let {vi} be a "degree-alternating" sequence of vertices. If we define a new 
sequence {S} of positive integers as: 

- For all k>0, s 2k =d~(v 2 k) 

- For all k > 0, s 2 k+i=d + (v 2 k+i), 

then this sequence of positive integers is a non-increasing sequence. 

Moreover, if Vi and Vi+i are two consecutive vertices in the sequence such that 
d~(v l+ i) ^ d + (v.i) then s i+ i < s,. 

Using this claim, we would finish the proof of Theorem [2] Let there be one vertex 
w S V such that d + (u) ^ d~ (w) for some edge (u, w) G ~e) . Let w' be the vertex such 



Since we have already argued that in the graph every connected component has to 
be strongly connected, we can create an infinite sequence of vertices such that w and w' 
appears consecutively and infinitely often. Now by Claim[3] it means that the sequence 
{S} is a non-increasing sequence that decreases infinitely many times. But this cannot 
happen as all the numbers in the sequence {S} represent in-degree or out-degree of 
vertices and hence, are always finite integers and can never be negative. Thus, if one 
vertex w G V such that d + (u) ^ d~(w) for some edge (u, w) £ £^ then we hit a 
contradiction. 

Thus, for all edges (u, v u) = d (v). □ 

Proof of Claim |3j This proof uses the fact that since the uniform distribution is a sta- 
tionary distribution for the Markov chain, for all vertices v 



and hence, we have d (v 2 i) > d + (v 2 i + i) which by definition gives s 2i > s 2 i + \. 



that (w\ w) £~f) and 



d + (w') = mm{d + (u) : (u,w) G i^}. 




Let us first prove that in the sequence {S}, s 2 i > s 2 i + \. 
Since d + (v 2l+ \) = min|d + (w) : (w,v 2i ) G itX, 




(1) 
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Now let us also prove that in the sequence {S}, Sai-i > s 2i- By definition, this 
is same as proving d + (v2i-i) > d~(v2i). Since we have assumed that the graph is a 
degree-Z\ graph, proving S2i-i > S2i is same as proving d~{v2i-i) < d + (v2i). 

Similar to previous case, since 

d + (v 2 t) = max ^d + (w) : (w, V21-1) G E j 

and 

1= y _^> rf ~(^-i) ( 2 ) 

(«,-u2»-i)e J5 

hence we have S2,_i > S2i- 

Note that the inequalities in Equation Q] and [2] are strict inequalities if d + (v2?:+i) 7^ 
d~(v2i) and d + (v2i-i) 7^ d~(v2i) respectively. Thus, if for any i, d + (vj + i) ^ d~{v.i) 
then Sj+i < Si. □ 

From Theorem|2]we can also obtain the following corollary. 

Corollary 1. Let ~& = (V, it) fee a connected degree- A oriented graph. Then the 
uniform distribution of vertices is a stationary distribution for the random walk markov 
chain on ~G, if and only if the following conditions apply: 

1. If G = (V, E) is non-bipartite, then the graph is Eulerian. 

2. If G is bipartite with bipartition V\ U V% = V then \V\\ = | V2 1 and in-degree of 
all vertices in one partition will be same and it will be equal to out-degree of all 
vertices in other partition. 

Proof: (Part 1) From Theorem [2] we follow that the uniform distribution of vertices 
is a stationary distribution for the random walk Markov chain on G iff for every edge 
(it, v) G E, we have 

d + (u) = d-(v). (3) 

In an Eulerian graph, all vertices will have same in-degree and out-degree (as all ver- 
tices in ~(% has the same total degree), hence the condition|3]is satisfied. And using The- 
orem[2] we conclude that the uniform distribution is a stationary distribution. Now, if G 
satisfies condition[3] then if there is an edge between vertex u and v (either (u, v) G E 
or (v, u) G E) then d~(v) = d + (u). Thus, in an undirected graph, if there is a path of 
odd length from u to v then d" (v) = d + (u). Thus, if the undirected graph has an odd 
cycle passing through v then d~(v) = d + (v). Note that, if there exist a vertex having 
in-degree equal to out-degree, then all the vertices in the connected component have 
in-degree equal to out-degree. This implies that every connected component that has an 
odd cycle must be Eulerian. 

(Part 2) If we start traversing from u (a vertex from left partition of bipartite graph 
~Cx) on underlying undirected graph G of G then all vertices which are at even length 
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distance from u will fall on left partition of G and will have out-degree d + (u). Now 
proof of Corollary followed by equation [5] □ 

Theorem[2]and Corollary Q] has an application to property testing. We present this 
application in next section. 



4 Application to Property Testing 

A property of a graph that is invariant under graph isomorphism is called as graph 
property. Testing of graph properties has been a very hot topic in the area of property 
testing (see Q, ID). Note that, whether the uniform distribution on the vertices of G 
is a stationary distribution is actually a graph property. When the input is a graph (as in 
our case), how to query the input is very important. Various models have been studied in 
this respect, for example: dense-graph models, sparse-graph models, orientation model 
etc. 

Thus, the problem of distinguishing between whether the stationary distribution of 
Markov chain is the uniform distribution or is "far" from it, is a question of testing 
graph properties. Here, this question has been framed in the orientation model (defined 
in the next subsection). Some interesting graph properties like connectedness Q] and 
Eulerianity have been studied in this model. Using Theorem Q] we show that for 
both bipartite and non-bipartite graphs, testing (in the orientation model) whether uni- 
form distribution on vertices of is the stationary distribution, (for the Markov chain 
generated by a lazy random walk on G^) can be reduced to testing if the graph is Eule- 
rian. Using the algorithms from of testing Eulerianity in the orientation model, we 
obtain various bounds on the query complexity for testing uniformity of the stationary 
distribution for the Markov chain generated by a random walk on C^. 

In O, it is shown that if G is an a-expander then whether G is Eulerian can be 
tested using O(Afa) number of queries in the orientation model. From our result, it 
implies that if G is an a-expander then testing uniformity of the stationary distribution 
can be done with 0(A/a) queries. Since 1/a is also a measure on the mixing time 
of the random walk, it implies that the query complexity for testing uniformity of the 
stationary distribution is directly proportional to the mixing time of the Markov chain. 



4.1 Property Testing in the Orientation Model 

Given an oriented graph = (V, i^) and a property P, we want to test whether 
satisfies the property or it is "e-far" from satisfying the property. In the orientation 
model, the underlying graph G = (V, E) is known in advance. Each edge in E is 
oriented (that is directed in exactly one direction). The orientation of the edges has to 
be queried. The graph is said to be "e-far" from satisfying the property P if one has to 
reorient at least e fraction of the edges to make the graph have the property. 
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The goal is to design an e-tester (that is, a randomized algorithm) that queries the 
orientation of the edges and does the following: 

- If ~& satisfies the property P then the tester ACCEPTS with probability 2/3, 

- If is "e-far" from satisfying the property P then the tester REJECTS with prob- 
ability 2/3. 

The query complexity of the algorithm is defined as the number of edges it queries. The 
natural goal is to design a tester for P with minimum query complexity. If the graph 
satisfies the property and the tester accepts with probability 1, then the tester is called a 
1-sided error tester. The usual tester is called a 2-sided error tester. 

The orientation model for testing graph properties was introduced by Halevy et al. 
|4|. In (3), Fischer et al. studied the problem of testing whether an oriented graph 
is Eulerian. They derived various upper and lower bounds for both 1-sided and 2-sided 
error testers. They even considered various special classes of graphs such as the ex- 
panders. In this paper, we use their algorithms for testing whether uniform distribution 
is the stationary distribution for the random-walk-Markov -chain. 

The reason of choosing orientation model (not any other graph model like dense, 
sparse model etc.) for property testing is because in this model, we need to know the 
underlying undirected graph in advance, which is a regular graph in our case. And we 
query orientation of edges in order to distinguish whether the input satisfies the property 
or is far from satisfying it, which in our case is to test whether the stationary distribution 
is uniform or is far from being uniform. 



4.2 Testing for Uniformity of Stationary Distribution in the Orientation Model 

Given a degree-Z\ oriented graph = (V, E), we say that the graph has the property 
V if for all (u, v) £ E, we have d + (u) — dT (v) . 

In Theorem[2] we proved that the uniform distribution is a stationary distribution for 
the random walk on a degree-Z\ oriented graph iff the graph G satisfies the property 
V. Thus, given a degree-Z\ oriented graph G, testing whether the uniform distribution 
is the stationary distribution for the random walk on , is same as testing if the graph 
has the property V. Thus, we want to test whether the graph G has the property V or is 
"far" from having the property. 

Note that the stationary distribution is uniform for the random walk on the whole 
graph iff the stationary distribution is uniform for the random walk on every con- 
nected component of G*. Since the undirected graph is known in advance, we have the 
connected components and hence, we need to test each component separately. Also, 
note that if the graph ~G* is "e-far" from satisfying the property V, then there is at least 
one connected component of that is also "e-far" from satisfying the property V. 
Thus, we can do testing connected-component wise and without loss of generality, we 
can assume that the graph is connected. 



Testing Uniformity of Stationary Distribution 9 



Let us assume G = (V, E) is connected. From Corollary [TJ if is non-bipartite 
then we have to test whether ~G is Eulerian. Since we can determine whether a con- 
nected component is bipartite or not just by looking at the underlying undirected graph, 
if ~& is non-bipartite then we use the testing algorithm from [3 j to test whether the graph 
is Eulerian. 

Now let be bipartite. Since is connected, there is a unique partition of the 
vertex set of ~^ that makes it bipartite and that partition can be found in the preprocess- 
ing phase just by looking at the underlying undirected graph. Let the bipartition be Vl 
and Vr. If \Vl\ ^ \Vr\ then the graph surely does not have uniform distribution as the 
stationary distribution for the random walk on ~&. By Corollary H] if \V L \ = \V R \ then 
the graph must have the property that the out-degree of all vertices in Vl must be equal 
to the in-degree of all vertices in Vr and vice versa. 

Let v be a vertex in Vl and d~(v) = ki and d + (v) = k 2 - Now consider any 
bipartite directed graph G^ = (V,Er) with bipartition Vl and Vr that satisfies the 
following conditions: 

- The underlying undirected graphs of C^andG^ are exactly same 

- For all v G Vl, (v) = k 2 and (v) = ki, and 

- For all v <G Vr, dZj. (v) = k\ and d\ (v) = k 2 . 

Note that if such graph G does not exist then it means that G cannot have the property 
V. If such a graph exists then consider the graph = (V, ~E* + E^) obtained by 
superimposing ~d and G*. Clearly, if ^ has the property V then G^ is Eulerian, and 
farness from having property V is also true by following lemma: 

Lemma l.If~$ is "e-far" from having property V then G^ is "—far" from being 
Eulerian. 

Proof: Let if be Eulerian graph which is the closest to ~G®. Since the underlying undi- 
rected graph for and G^ are exactly same, there is one-to-one correspondence be- 
tween the edges in andET. 

Now, look at edges of G^ that were flipped in order to obtain if. Suppose a flipped 
edge (say e) belonged to E^. Then, we can re-flip this edge e and flip the corresponding 
edge in if. Thus, we have effectively flipped the same number of edges. By performing 
these operations on flipped edges of E , we have obtained a new graph which has the 
same number of flipped edges as G^ and all the flipped edges in G^ belong to if. 

Thus, if the graph G^ is not "|-far" from being Eulerian then is not "e-far" from 
having property V, which is a contradiction. □ 

Now, all we have to test is whether the new graph G^ is Eulerian or "§-far" from 

being Eulerian. Note that every query to G® can be simulated by a single query to 
G. Thus, we can now use the Eulerian testing algorithm from J3]. The algorithm is 
summarized in Algorithm 1 and the various bounds on the query complexity that can 
be obtained is summarized in Table 1. 
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Data: Degree-zl Oriented Graph = (V, E) 

Result: Whether has the property V or is "e-far" from having it. 

1 if G is non-bipartite then 

2 I Test if ~d is Eulerian (see |3)) and give the corresponding output. 

3 else 

4 Let Vl and Vr be the bipartition for the graph G. 

5 Sample a vertex from Vl and query all edges incident on it. Let d~ (v) = k\ 
and d + (v) = k2- 

Construct any bipartite graph G* = (V, E*) with bipartition (Vl, Vr) such 
that 

(a) For all v 6 Vl, gC^ (w) = fc2 and <f±^ (v) = k\, and for all v E Vr, 
(v) = ki and (v) = k2- 

(b) The underlying graph of G* is exactly same as G — (V, E). 
Superimpose (?* and graph G (say G®=(V, E + E 1 ). 
Test if G^ is Eulerian (see El) and give the corresponding output. 

it end 



Algorithm 1: Algorithm for testing property V 



1 -sided test 



2-sided test 



Graphs with large A 



T 



Bounded-degree graphs * 



Q [mi 



n 



log m 
log log m 



a-expander 



O 



zilog(f 



min < O 



Mi) 



,0 



^3 log 



* Lower bound holds for 4-regular graph. 

A is the maximum degree of the underlying undirected graph and m is the number of edges is 
the graph. 

Table 1. Bounds on the query complexity (in the orientation model) for testing unifor- 
mity of stationary distribution of the Markov chain obtained by the random walk on a 
directed graph. 
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5 Conclusion 

We showed that, for a given degree- A oriented graph G = (V,E), whether the uniform 
distribution on vertices of ~G* is a stationary distribution, depends on a local property of 
graph. If ~G satisfies this local property then it has some particular kind of structure 
(from Corollary [T). Finally, as an application of this result, we showed that testing this 
local property in orientation model, can be reduced to testing Eulerianity (see J3|). 

The result holds only for graphs where the in-degree plus out-degree of all the ver- 
tices are the same. It would be interesting to see if one can make a similar statement for 
more general graphs. 
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